Upcoming Event: Babuška Forum

Global Optimization

Bjorn Engquist,

Abstract

To be concrete we will focus on minimization and with global we mean minimization where the objective function may not be convex and can have several local minima. We will also deal with unconstrained minimization. This field has seen drastically increasing attention since the loss function in machine learning is typically far from convex. Standard optimization methods as, for example, gradient descent and Newton’s method will converge to local minima, which may not be the global minimum.

After surveying classical methods for global optimization, such as simulated annealing, we will discuss recent development of stochastic gradient descent methods and related convergence theory. We will see that having an adaptive state dependent stochastic term is beneficial. Depending on the setting, convergence in probability or almost sure convergence is possible.

Biography

Bjorn Engquist received his Ph.D. in numerical analysis from Uppsala University in 1975. He has been Professor of Mathematics at UCLA, and the Michael Henry Stater University Professor of Mathematics and Applied and Computational Mathematics at Princeton University. He was director of the Research Institute for Industrial Applications of Scientific Computing and of the Centre for Parallel Computers at the Royal Institute of Technology, Stockholm. At Princeton University, he was director of the Program in Applied and Computational Mathematics and the Princeton Institute for Computational Science. Engquist came to The University of Texas at Austin in 2004, where he is Professor of Mathematics holding the Computational and Applied Mathematics Chair I, and is Director of the Oden Institute Center for Numerical Analysis. Engquist is a member of the Royal Swedish Academy of Sciences, the Royal Swedish Academy of Engineering Sciences and the Norwegian Academy of Science and Letters. He is also a member of the American Academy of Arts and Sciences and a SIAM Fellow.

Engquist’s research focuses on development, analysis and application of numerical methods for differential equations. His earlier work includes the development of absorbing or far field boundary conditions, homogenization theory and nonlinear high-resolution schemes for compressible fluid dynamics. Application areas have been acoustic and electromagnetic wave propagation, aerodynamics and flow in porous media. More recently he has been working on computational multi-scale methods and in particular the development of the Heterogeneous Multi-scale Method. Another recent focus is fast algorithms for wave propagation with applications in seismology. The application to seismology includes novel algorithms for seismic inversion.